Traffic information providing method and device, and computer program stored in medium in order to execute method

ABSTRACT

An object tracking method using traffic information, includes the steps of: receiving a travel route request for a first vehicle from a terminal; and allowing a database, in which a record for each of a plurality of frames included in images captured by a plurality of cameras for photographing traffic information is stored and includes an object tag and an event tag recognized in each frame, to provide, while searching for the record in chronological order, route information including photographing position information and/or images corresponding to a continuous first record section including an object tag corresponding to first vehicle information, a second record including an object tag and a stopover event tag corresponding to the first vehicle information, and a continuous third record section, which is continuous to the second record and includes an object tag corresponding to first person information included in the stopover event tag of the second record.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a Continuation of U.S. application Ser. No. 16/648,667 filedMar. 19, 2020, which is a National Stage Entry of International PatentApplication No. PCT/KR2017/012462, filed on Nov. 6, 2017, and claimspriority from and the benefit of Korean Patent Application No.10-2017-0140812, filed on Oct. 27, 2017, each of which is herebyincorporated by reference for all purposes as if fully set forth herein.

BACKGROUND Field

Exemplary embodiments of the invention relate to a traffic informationproviding method and apparatus, and a computer program stored in amedium to execute the method.

Discussion of the Background

Closed-circuit televisions (CCTVs) have been installed in many zones,such as the express ways, general roads, and residential areas, to beused to find criminals, suspects, and vehicles involved in crimes incase of need. A method of recording video to look for a person orvehicle from the recorded video as necessary has been primarily used.However, because there is no information about the correlation betweenobjects, it is difficult to track objects although vehicle recognition,face recognition and other various image analysis algorithms, etc., areused.

SUMMARY

Provided are a traffic information providing method and apparatus forproviding an object tracking service of associating information aboutvehicles with information about persons, and a computer program storedin a medium to execute the method. Also, provided is a trafficinformation processing method of establishing a database capable ofassociating information about vehicles with information about persons.

Also, provided are a computer program for providing the method, and acomputer-readable recording medium storing the computer program. Itshould be noted that technical objects of the present embodiments arenot limited to the above-described technical objects, and othertechnical objects will be apparent to those skilled in the art from thefollowing embodiments.

An embodiment of the disclosure provides an object tracking method usingtraffic information, including: receiving a request for a travel routeof a first vehicle from a terminal; and searching for a record inchronological order in a database storing a plurality of recordsrespectively for a plurality of frames included in images captured by aplurality of cameras photographing traffic information, each recordincluding an object tag and an event tag recognized from a frame, andproviding route information including at least one of an image andphotographing location information corresponding to a continuous firstrecord section including an object tag corresponding to informationabout the first vehicle, a second record including a getting-off eventtag and the object tag corresponding to the information about the firstvehicle, and a continuous third record section following the secondrecord and including an object tag corresponding to information about afirst person included in the getting-off event tag of the second record.

The method may further include: an object tag generating operation ofrecognizing at least one of a person and a vehicle from a frame includedin the images captured by the plurality of cameras photographing thetraffic information, and generating an object tag for the frame; and anevent tag generating operation of referring to the object tag, and whena vehicle and a person within a predetermined distance from the vehicleare recognized from a first frame, and the vehicle is recognized whilethe person is not recognized from a second frame following the firstframe, generating a getting-in event tag including information about thevehicle and information about the person for the second frame, or when avehicle is recognized while no person is recognized from a third frameand the vehicle and a person within the predetermined distance from thevehicle are recognized from a fourth frame following the third frame,generating a getting-off event tag including information about thevehicle and information about the person for the fourth frame.

The object tag generating operation includes sampling the image into aplurality of sampled frames at an interval N of N frames, determiningwhether at least one of a person and a vehicle is present for each ofthe plurality of sampled frames, and generating a plurality of objecttags for the plurality of frames, respectively, according to thedetermined result, and the event tag generating operation includesgenerating a tag of at least one of a getting-in event and a getting-offevent with reference to the plurality of object tags generated for theplurality of sampled frames, wherein the first frame and the secondframe are successive in the plurality of sampled frames, and the thirdframe and the fourth frame are successive in the plurality of sampledframes.

The object tag generating operation may include including an object tagrecognized from each frame of the plurality of frames included in theimages in a record for the frame, and storing the record in thedatabase, and the event tag generating operation may include generatingthe event tag with reference to the object tag of the record, andstoring the event tag in a record of a frame corresponding to the eventtag.

The event tag generating operation may include, when the vehicle and theperson within the predetermined distance from the vehicle are recognizedfrom the first frame, the vehicle is recognized while the person is notrecognized from the second frame following the first frame, and theperson is not recognized from m frames after the second frame,generating the getting-in event tag including the information about thevehicle and the information about the person for the second frame.

The event tag generating operation may include, when the vehicle and theperson within the predetermined distance from the vehicle are recognizedfrom the first frame, the vehicle is recognized while the person is notrecognized from the second frame following the first frame, and theperson is not recognized from m frames after a photographing time of thesecond frame photographed by an adjacent camera of a camera that hasphotographed the second frame, generating the getting-in event tagincluding the information about the vehicle and the information aboutthe person for the second frame.

Another embodiment of the disclosure provides an object tracking methodusing traffic information, including: receiving a request for a travelroute of a first person from a terminal; and

searching for a record in chronological order in a database storing aplurality of records respectively for a plurality of frames included inimages captured by a plurality of cameras photographing trafficinformation, each record including an object tag and an event tagrecognized from a frame, and providing route information including atleast one of an image and photographing location informationcorresponding to a continuous first record section including an objecttag corresponding to information about the first person, a second recordincluding a getting-in event tag and the object tag corresponding to theinformation about the first person, and a continuous third recordsection following the second record and including an object tagcorresponding to second vehicle information included in the getting-inevent tag of the second record.

Another embodiment of the disclosure provides a computer program storedin a medium to execute at least one method described above by using acomputer.

Another embodiment of the disclosure provides an object trackingapparatus using traffic information, including: a communicatorconfigured to receive a request for a travel route of a first vehiclefrom a terminal; and a route provider configured to search for a recordin chronological order in a database storing a plurality of recordsrespectively for a plurality of frames included in images captured by aplurality of cameras photographing traffic information, each recordincluding an object tag and an event tag recognized from a frame, andprovide route information including at least one of an image andphotographing location information corresponding to a continuous firstrecord section including an object tag corresponding to informationabout the first vehicle, a second record including a getting-off eventtag and the object tag corresponding to the information about the firstvehicle, and a continuous third record section following the secondrecord and including an object tag corresponding to information about afirst person included in the getting-off event tag of the second record.

The object tracking apparatus may further include: an object recognizerconfigured to recognize at least one of a person and a vehicle from aframe included in the images captured by the plurality of camerasphotographing the traffic information, and generate an object tag forthe frame; and an event recognizer configured to refer to the objecttag, and when a vehicle and a person within a predetermined distancefrom the vehicle are recognized from a first frame and the vehicle isrecognized while the person is not recognized from a second framefollowing the first frame, generate a getting-in event tag includinginformation about the vehicle and information about the person for thesecond frame, or when a vehicle is recognized while no person isrecognized from a third frame and the vehicle and a person within thepredetermined distance from the vehicle are recognized from a fourthframe following the third frame, generate a getting-off event tagincluding the information about the vehicle and the information aboutthe person for the fourth frame.

The object recognizer may be further configured to sample the image intoa plurality of sampled frames at an interval N of N frames, determinewhether at least one of a person and a vehicle is present for each ofthe plurality of sampled frames, and generate a plurality of object tagsfor the plurality of frames, respectively, according to the determinedresult, and the event recognizer may be further configured to generate atag of at least one of a getting-in event and a getting-off event withreference to the plurality of object tags generated for the plurality ofsampled frames, wherein the first frame and the second frame aresuccessive in the plurality of sampled frames, and the third frame andthe fourth frame are successive in the plurality of sampled frames.

The object recognizer may be further configured to include an object tagrecognized from each frame of the plurality of frames included in theimage in a record for the frame, and store the record in the database,and the event recognizer may be further configured to generate the eventtag with reference to the object tag of the record, and store the eventtag in a record of a frame corresponding to the event tag.

The event recognizer may be further configured to, when the vehicle andthe person within the predetermined distance from the vehicle arerecognized from the first frame, the vehicle is recognized while theperson is not recognized from the second frame following the firstframe, and the person is not recognized from m frames after the secondframe, generate the getting-in event tag including the information aboutthe vehicle and the information about the person for the second frame.

The event recognizer may be further configured to, when the vehicle andthe person within the predetermined distance from the vehicle arerecognized from the first frame, the vehicle is recognized while theperson is not recognized from the second frame following the firstframe, and the person is not recognized from m frames after aphotographing time of the second frame photographed by an adjacentcamera of a camera that has photographed the second frame, generate thegetting-in event tag including the information about the vehicle and theinformation about the person for the second frame.

Another embodiment of the disclosure provides an object trackingapparatus using traffic information, including: a communicatorconfigured to receive a request for a travel route of a first personfrom a terminal; and a route provider configured to search for a recordin chronological order in a database storing a plurality of recordsrespectively for a plurality of frames included in images captured by aplurality of cameras photographing traffic information, each recordincluding an object tag and an event tag recognized from a frame, andprovide route information including at least one of an image andphotographing location information corresponding to a continuous firstrecord section including an object tag corresponding to informationabout the first person, a second record including a getting-in event tagand the object tag corresponding to the information about the firstperson, and a continuous third record section following the secondrecord and including an object tag corresponding to second vehicleinformation included in the getting-in event tag of the second record.

According to an embodiment of the disclosure, because a database capableof associating information about vehicles with information about personsis established, and a tracking function of associating information aboutvehicles with information about persons is provided by using theestablished database, it is possible to track a suspect or a suspectedvehicle while keeping sight of the suspect or the suspected vehicle evenwhen a person makes a getaway after getting in a vehicle or aftergetting off a vehicle.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 shows a system according to an embodiment of the disclosure.

FIG. 2 is a block diagram schematically showing an object trackingdevice of a traffic information providing apparatus according to anembodiment of the disclosure.

FIG. 3 shows an example of a table stored in a database according to anembodiment of the disclosure.

FIG. 4 shows another example of a table stored in a database accordingto an embodiment of the disclosure.

FIG. 5 shows an example of a frame in which a getting-in event occurs.

FIG. 6 shows an example of a frame in which a getting-off event occurs.

FIG. 7 shows another example of a table stored in a database, accordingto an embodiment of the disclosure.

FIG. 8 shows another example of a table stored in a database, accordingto an embodiment of the disclosure.

FIG. 9 is a flowchart showing a traffic information processing methodaccording to an embodiment of the disclosure.

FIG. 10 is a flowchart showing an object tracking method according to anembodiment of the disclosure.

DETAILED DESCRIPTION

The disclosure can have various modifications and various embodiments,and specific embodiments are shown in the drawings and described in thedetailed description, although the disclosure is not limited to thespecific embodiments. That is, it has to be understood that thedisclosure includes all modifications, equivalents, and substitutesincluded in the concept and technical scope of the disclosure. In thefollowing description, if it is determined that detailed descriptionsfor related art make the subject matter of the disclosure obscureunnecessarily, the detailed descriptions will be omitted.

It will be understood that, although the terms “first”, “second”, etc.may be used herein to describe various components, these componentsshould not be limited by these terms These terms are only used todistinguish one component from another.

The terms used in the present specification are used to describe thespecific embodiments, not for the purpose of limiting the disclosure. Itis to be understood that the singular forms “a,” “an,” and “the” includeplural referents unless the context clearly dictates otherwise. It willbe understood that when the terms “includes,” “comprises,” “including,”and/or “comprising,” when used in this specification, specify thepresence of stated features, figures, steps, components, or combinationthereof, but do not preclude the presence or addition of one or moreother features, figures, steps, components, members, or combinationsthereof.

Hereinafter, the disclosure will be described in detail with referenceto embodiments of the disclosure shown in the accompanying drawings.

FIG. 1 shows a system according to an embodiment of the disclosure.

Referring to FIG. 1 , a system according to an embodiment of thedisclosure may include a plurality of cameras C for photographingtraffic information, and a server S for receiving images from thecameras C, storing the images, and providing a tracking service based onthe stored images.

The system according to an embodiment of the disclosure may furtherinclude a terminal T that requests the tracking service and receives thetracking service.

The cameras C, the server S, and the terminal T may transmit and receivedata through a network N.

The cameras C may photograph places where vehicles and/or persons arepresent, such as roads, sidewalks, parking lots, streets, etc. Two ormore cameras C may be installed at different locations. Each camera Cmay be a pan tilt zoom (PTZ) camera capable of panning, tilting, andzooming. The camera C may transmit a captured image to the server Sthrough the network N.

The server S may receive the image from the camera C, analyze the image,and store the image and the analyzed result. The server S may controlthe camera C. The server S may provide at least one service to theterminal T, which is a client. For example, the server S may provide anobject tracking service to the terminal T.

The terminal T may request the server S to look up stored data. Theterminal T may request the server S to look up a stored image. Theterminal T may request the server S to track a specific object based onthe stored data. The terminal T may transmit information about thespecific object to the server S to request the server S to track atravel route of the corresponding object, and receive a tracked routefrom the server S.

The terminal T may be a personal computer (PC), a smart phone, a tabletcomputer, a laptop computer, or the like, and may include all kinds ofterminals that are capable of accessing a web/mobile site related to theserver S and installing and executing a service-dedicated program. Aplurality of terminals T may be included in the system.

The network N may include a wired network, such as, for example, localarea networks (LANs), wide area networks (WANs), metropolitan areanetworks (MANs), integrated service digital networks (ISDNs), or thelike, and a wireless network, such as wireless LANs, code divisionmultiple access (CDMA), Bluetooth, 3^(rd) Generation (3G), 4^(th)Generation (4G), 5th Generation (5G), long term evolution (LTE), or thelike, although not limited thereto.

FIG. 2 is a block diagram schematically showing an object trackingdevice 100 according to an embodiment of the disclosure. The objecttracking device 100 may be installed in the server S.

Referring to FIG. 2 , the object tracking device 100 may include aprocessor 110, a database 120, and a communicator 130. In the objecttracking device 100 shown in FIG. 2 , only components related to thecurrent embodiment are shown to prevent a characteristic of the currentembodiment from becoming obscure. Accordingly, it will be understood byone of ordinary skill in the art that other general-purpose componentsthan the components shown in FIG. 2 may be additionally included.

The processor 110 shown in FIG. 2 may correspond to or include at leastone processor. Accordingly, the object tracking device 100 may beincluded in another hardware device, such as a microprocessor or ageneral-purpose computer system, and driven. The processor 110 mayperform data processing required for the embodiments of the disclosureaccording to execution of a program in a memory (not shown).

The embodiments of the disclosure may be represented by functional blockconfigurations and various processing operations of the processor 110.Functional aspects may be implemented as algorithms that are executed byone or more processors.

The processor 110 may include an object recognizer 111, an eventrecognizer 112, and a route provider 113.

The database 120 may store images received from the camera C or imageinformation analyzed by the processor 110.

The communicator 130 may transmit and receive data to and from thecamera C and/or the terminal T. The communicator 130 may receive animage from the camera C, receive an object tracking request from theterminal T, and provide a result of object tracking to the terminal T.

According to an embodiment, the communicator 130 may receive images fromthe plurality of cameras C that photograph traffic information.Alternatively, the communicator 130 may receive images captured by theplurality of cameras C that photograph traffic information, from anotherserver. According to an embodiment, the object recognizer 111 mayrecognize at least one of a person and a vehicle from a frame includedin an image. The object recognizer 111 may generate an object tag forthe frame. The object tag may be a tag for a person or a tag for avehicle, and may include identification information for identifying anobject. For example, information for identifying a person may includeface recognition information, a height, information about whetherglasses are present, gender information, silhouette information,biometric information, or personal information (for example, a residentregistration number or a cell phone number), and information foridentifying a vehicle may include at least one of a license platenumber, a model, a color, and a vehicle registration number. Theidentification information may be acquired from an image by the objectrecognizer 111.

According to an embodiment, the object recognizer 111 may perform objectrecognition and tag generation on all frames included in an image.

According to an embodiment, the object recognizer 111 may sample framesincluded in an image at a sampling interval N of N frames, determinewhether at least one of a person and a vehicle is present for each ofthe sampled frames, and generate an object tag for each of the sampledframes according to the determined result. The sampling interval N maybe set by a manager, or appropriately set considering the performance ofthe processor 110 and a frame rate of the image. For example, thesampling interval N may be set to the frame rate so that a time intervalof the sampled frames becomes 1 second.

The event recognizer 112 may generate a getting-in event and/or agetting-off event with reference to the object tag generated by theobject recognizer 111. More specifically, when a vehicle and a personwithin a predetermined distance from the vehicle are recognized from afirst frame, and the vehicle is recognized while the person is notrecognized from a second frame following the first frame, the eventrecognizer 112 may determine that the person has gotten into thevehicle, and generate a getting-in event tag including information aboutthe vehicle and information about the person for the second frame.

When a vehicle is recognized while no person is recognized from a thirdframe and the vehicle and a person within a predetermined distance fromthe vehicle are recognized from a fourth frame following the thirdframe, the event recognizer 112 may determine that the person has gottenoff the vehicle, in the fourth frame, and accordingly, the eventrecognizer 112 may generate a getting-off event tag for the fourthframe. The getting-off event tag may include information about theperson who has gotten off and information about the vehicle which theperson has gotten off.

As described above, the object recognizer 111 may sample an image into aplurality of sampled frames, and the event recognizer 112 may alsogenerate a tag of at least one of a getting-in event and a getting-offevent for the plurality of sampled frames with reference to object tagsgenerated for the plurality of sampled frames. In this case, the firstframe and the second frame as described above may be successive framesof the plurality of sampled frames, and the third frame and the fourthframe may be successive frames of the plurality of sampled frames.

The object recognizer 111 according to an embodiment may generate arecord for each of a plurality of frames (or each sampled frame)included in an image, and store the record in the database 120. Therecord may include an object tag recognized from the frame. The objecttag may include object identification information.

The event recognizer 112 according to an embodiment may generate anevent tag described above with reference to the object tag of the recordstored in the database 120, include the generated event tag in a recordof a frame corresponding to the event tag, and store the record in thedatabase 120.

In the above-described embodiments, because an object tag is generatedonly when an object exists for each frame and an event tag is generatedonly when an event occurs, there may be a frame including neither anobject tag nor an event tag, a frame including only an object tag, aframe including only an event tag, and a frame including both an objecttag and an event tag.

According to an embodiment, the communicator 130 may receive a requestfor a travel route of a first vehicle from the terminal T. The routeprovider 113 may track a travel route of the first vehicle in thedatabase 120, and the travel route tracked by the route provider 113 maybe provided to the terminal T by the communicator 130.

More specifically, the route provider 113 may search for a plurality ofrecords stored in the database 120 in chronological order and provideroute information including at least one of an image and photographinglocation information corresponding to a continuous first record sectionincluding an object tag corresponding to information about the firstvehicle, a second record including a getting-off event tag and theobject tag corresponding to the information about the first vehicle, anda continuous third record section following the second record andincluding an object tag corresponding to information about a firstperson included in the getting-off event tag of the second record.

When a request for a travel route of a person is received from theterminal T, the route provider 113 may provide a travel route of theperson by replacing the person with a vehicle and the vehicle with aperson in the above-described embodiment.

For example, the communicator 130 may receive a request for a travelroute of the first person from the terminal T. In this case, the routeprovider 113 may search for a plurality of records stored in thedatabase 120 in chronological order and provide route informationincluding at least one of an image and photographing locationinformation corresponding to a continuous first record section includingan object tag corresponding to information about the first person, asecond record including a getting-in event tag and the object tagcorresponding to the information about the first person, and acontinuous third record section following the second record andincluding an object tag corresponding to information about a secondvehicle included in the getting-in event tag of the second record.

FIG. 3 shows an example of a table stored in the database 120 accordingto an embodiment of the disclosure.

Each of records 31 to 36 shown in FIG. 3 may correspond to a frame.Referring to FIG. 3 , each of the records 31 to 36 may include data fora plurality of columns. Each column may include channel number No.,time, and tag information. Referring to the record 31, a framecorresponding to the record 31 may have been photographed at 01:01:01 bya camera corresponding to channel 1, and a person “A” may be recognizedfrom the corresponding frame by the object recognizer 111, whichrepresents that person information has been tagged. Referring to therecord 35, a frame corresponding to the record 35 may have beenphotographed at 01:05:03 by a camera corresponding to channel 2, and theperson “A” and a vehicle “B” may be recognized from the correspondingframe by the object recognizer 111, which represents that personinformation and vehicle information have been tagged.

Channel numbers of cameras may be replaced by location information(coordinates, addresses, etc.) of the cameras. Alternatively, thelocation information of the cameras corresponding to the channel numbersof the cameras may be stored together with the channel numbers of thecameras in the database 120, or may be set to be looked up by the serverS.

Meanwhile, the table shown in FIG. 3 may be a table stored in thedatabase 120 which stores object tags for a vehicle and person of eachframe, recognized and generated by the object recognizer 111.

FIG. 4 shows another example of a table stored in the database 120according to an embodiment of the disclosure.

The table shown in FIG. 4 may be a table obtained by adding a tag of anevent recognized by the event recognizer 112 with reference to the tableshown in FIG. 3 to each record.

Referring to FIG. 4 , the event recognizer 112 may refer to recordsstored in the database 120 in chronological order, and, when the eventrecognizer 112 determines that the record 32 in which both the person“A” and the vehicle “B” are tagged and the record 33 in which only thevehicle “B” is tagged are successive, the event recognizer 112 maydetermine that the person “A” has gotten in the vehicle “B”, and add agetting-in event tag to the record 33. The getting-in event tag added tothe record 33 may include identification information of “A” asinformation about a person which has gotten in a vehicle, andidentification information of “B” as information about a vehicle which“A” has gotten in.

According to an embodiment, a person tag and a vehicle tag may includeinformation about locations at which the corresponding objects have beenrecognized. In the current embodiment, the event recognizer 112 maydetermine a condition of whether a distance between a person and avehicle is shorter than a predetermined distance in a framecorresponding to the record 32 to recognize a getting-in event, and whenthe condition is satisfied, the event recognizer 112 may generate andstore a getting-in event tag. The predetermined distance may be setaccording to a sampling interval of frames. As the sampling interval offrames is longer, the predetermined distance may be set to a longerdistance.

The event recognizer 112 may refer to the records stored in the database120 in chronological order, and, when the event recognizer 112determines that the record 34 in which the vehicle “B” is tagged and therecord 35 in which the person “A” is tagged are successive in thisorder, the event recognizer 112 may determine that the person “A” hasgotten off the vehicle “B”, and add a getting-off event tag to therecord 35. The getting-off event tag added to the record 35 may includeidentification information of “A” as information about a person whichhas gotten off, and identification information of “B” as informationabout a vehicle which “A” has gotten off.

As described above, the event recognizer 112 may determine a conditionof whether a distance between a person and a vehicle is shorter than apredetermined distance in a frame corresponding to the record 35 torecognize a getting-off event, and, when the condition is satisfied, theevent recognizer 112 may generate and store a getting-off event tag. Thepredetermined distance may be set according to a sampling interval offrames. As the sampling interval of frames is longer, the predetermineddistance may be set to a longer distance.

In FIG. 4 , an example in which a tag for a person and a tag for avehicle are displayed is shown. However, the example is to describe anembodiment of the disclosure, and the disclosure is not limited to theexample. For example, when persons “A” and “C” are tagged in the record32 and the person “C” is tagged in the record 33, the event recognizer112 may generate and store a getting-in event because the eventrecognizer 117 determines that the person “A” is located around thevehicle “B” from the record 32 and the person “A” is not tagged in therecord 33 following the record 32.

When a plurality of vehicles are tagged in the records 32 and 33, theevent recognizer 112 may generate a getting-in event for the record 33such that the record 32 includes vehicle information recognized at alocation that is closest to the person “A”. When a plurality of vehiclesare tagged in the records 34 and 35, the event recognizer 112 maygenerate a getting-off event for the record 35 such that the record 35includes vehicle information recognized at a location that is closest tothe person “A”.

FIG. 5 shows an example of a frame in which a getting-in event occurs.

In FIG. 5 , a first frame 51 may correspond to the record 32 of FIG. 4 ,and a second frame 52 may correspond to the record 33 of FIG. 4 .

Referring to FIG. 5 , when the person “A” is not recognized from thesecond frame 52 following the first frame 51 from which the person “A”and the vehicle “B” are recognized, and the person “A” and the vehicle“B” are recognized within a predetermined distance from the first frame51, a getting-in event tag for the person “A” and the vehicle “B” may begenerated for the second frame 52.

FIG. 6 shows an example of a frame in which a getting-off event occurs.

In FIG. 6 , a third frame 61 may correspond to the record 34 of FIG. 4 ,and a fourth frame 62 may correspond to the record 35 of FIG. 4 .

Referring to FIG. 6 , when the person “A” is not recognized and thevehicle “B” is recognized from the third frame 61, and the person “A”and the vehicle “B” are recognized within a predetermined distance fromthe fourth frame 62 following the third frame 61, a getting-off eventtag for the person “A” and the vehicle “B” may be generated for thefourth frame 62.

FIG. 7 shows another example of a table stored in the database 120,according to an embodiment of the disclosure. Referring to FIG. 7 , aplurality of records 71 to 77 are shown, and the records 71 to 77 maycorrespond to successive frames.

Referring to FIG. 7 , the event recognizer 112 may perform validation togenerate a getting-in event. For example, before a getting-in event tagfor a record 73 is added according to a getting-in event generationcondition as described above, the event recognizer 112 may performvalidation for determining whether the person “A” included in thegetting-in event is not recognized from m records following the record73 to which the getting-in event tag is to be added. That is, when thevehicle “B” is recognized and the person “A” is recognized within apredetermined distance from the vehicle “B” from a first framecorresponding to the record 72, the vehicle “B” is recognized and theperson “A” is not recognized from a second frame corresponding to therecord 73, and the person “A” is not recognized from m records 74, . . .following the record 73, the event recognizer 112 may add a getting-inevent tag including information about the vehicle “B” and the person “A”to the record 73 and store the record 73.

According to an embodiment, the event recognizer 112 may additionallydetermine whether the person “A” is not recognized from m records aftera photographed time 01:01:03 of the record 73, in a channel (forexample, channel 2 being adjacent to channel 1 being a channel of therecord 73) being adjacent to the record 73 to which a getting-in eventtag is to be added for validation. When the person “A” is not recognizedin the adjacent channel, the event recognizer 112 may add a getting-inevent tag including information about the vehicle “B” and the person “A”to the record 73, and store the record 73.

According to an embodiment, when the person “A” gets out of aphotographing area of the channel 1 to disappear from an image and thenagain appears in a photographing area of the channel 2, no getting-inevent may be generated in the record 73. Therefore, when the person “A”is present around the vehicle “B” and then moves to get out of thephotographing area of the channel 1 to disappear accidently from aroundthe vehicle B, wrong recognition of determining that the person “A” hasgotten in the vehicle “B” may be prevented.

FIG. 8 shows another example of a table stored in the database 120,according to an embodiment of the disclosure. Referring to FIG. 8 , aplurality of records 81 to 87 are shown, and the records 81 to 87 maycorrespond to frames.

Referring to FIG. 8 , the event recognizer 112 may perform validation togenerate a getting-off event. For example, before a getting-off eventtag for a record 82 is added according to a getting-off event generationcondition as described above, the event recognizer 112 may performvalidation for determining whether the person “A” included in thegetting-off event is continuously recognized from m records followingthe record 82 to which the getting-off event tag is to be added. Thatis, when the vehicle “B” is recognized from a third frame correspondingto the record 81, the vehicle “B” and the person “A” within apredetermined distance from the vehicle “B” are recognized from a fourthframe corresponding to the record 82, and the person “A” is continuouslyrecognized from m records 83, . . . following the record 82, the eventrecognizer 112 may add a getting-off event tag including informationabout the vehicle “B” and the person “A” to the record 82 and store therecord 82.

FIG. 9 is a flowchart showing a traffic information processing methodaccording to an embodiment of the disclosure. The flowchart shown inFIG. 9 may be configured with operations that are processedtime-sequentially by the system shown in FIG. 1 and the object trackingdevice 100 shown in FIG. 2 . Therefore, all descriptions given abovewith reference to the configuration shown in FIG. 2 may be applied tothe flowchart shown in FIG. 9 , although some of the descriptions areomitted below.

Referring to FIG. 9 , in operation 91, the communicator 130 may receivean image from a camera C. Or, in operation 91, the communicator 130 mayreceive an image captured by the camera C from another server.

In operation 92, the object recognizer 111 may recognize at least one ofa person and a vehicle from a frame included in the image to generate anobject tag. The object recognizer 111 may store the generated object tagin the database 120.

In operation 93, the event recognizer 112 may generate a getting-inevent tag and/or a getting-off event tag, with reference to the objecttag generated in operation 92. The event recognizer 112 may store thegetting-in event tag and/or the getting-off event tag in the database120.

According to an embodiment of the disclosure, the traffic informationprocessing method shown in FIG. 9 may be designed to be performed by thecamera C. For example, in operation 91, the camera C may capture animage, and in operations 92 and 93, the camera C may itself analyze thecaptured image and generate an object tag and an event tag, and directlystore the generated tags in the database 120 of the server S or transmitthe tags to the server S to store the tags in the database 120. Themodifications of the embodiments may be designed according to computingperformance of a data processor included in the camera C. That is, whenthe computing performance of the data processor included in the camera Cis high, a tag generation operation may be processed by the camera C,and, otherwise, the tag generation operation may be processed by theserver S.

According to an embodiment of the disclosure, records stored in thedatabase 120 by the method of FIG. 9 may be collated regularly orirregularly to be backed up in a separate central database (not shown).

FIG. 10 is a flowchart showing an object tracking method according to anembodiment of the disclosure. Because the flowchart shown in FIG. 10 isconfigured with operations that are processed time-sequentially by theobject tracking device 100 shown in FIG. 2 , all the descriptions givenabove with reference to the configuration shown in FIG. 2 may be appliedto the flowchart shown in FIG. 10 , although some of the descriptionsare omitted below.

Referring to FIG. 10 , in operation 101, the communicator 130 mayreceive a request for tracking a route of the first vehicle from theterminal T.

In operation 102, the route provider 113 may extract a record includingan object tag corresponding to information about the first vehicle,while referring to records stored in the database 120 in chronologicalorder. Whenever a record in which the information about the firstvehicle is tagged is extracted, the route provider 113 may determinewhether a getting-off event for the first vehicle has been tagged in therecord, in operation 103. When the getting-off event has been tagged inthe record, the process may proceed to operation 104, and, when nogetting-off event has been tagged in the record, the process may proceedto operation 102 to extract the next record. When operation 102 isrepeatedly processed, a continuous first record section including theobject tag corresponding to the information about the first vehicle maybe extracted.

When an object tag corresponding to the information about the firstvehicle is no longer found, the process may proceed to operation 107from operation 102.

In operation 103, a second record in which a getting-off event is taggedmay be extracted.

In operation 104, tracking may continue to be performed by usinginformation about a first person included in the getting-off event tag.That is, the route tracking request from the terminal T relates to thefirst vehicle, however, according to an embodiment of the disclosure,the first person that has gotten off the first vehicle may continue tobe tracked. In operation 104, the route provider 113 may extract arecord including the object tag corresponding to the information aboutthe first person, while continuously referring to the records inchronological order. Whenever a record in which the information aboutthe first person is tagged is extracted, the route provider 113 maydetermine whether a getting-in event for the first person has beentagged in the record, in operation 105. When the getting-in event forthe first person has been tagged in the record, the process may proceedto operation 106, and otherwise, the process may again proceed tooperation 104 to extract the next record. When operation 104 isrepeatedly processed, a continuous third record section including theobject tag corresponding to the information about the first person maybe extracted.

When an object tag corresponding to the information about the firstperson is no longer found, the process may proceed to operation 107 fromoperation 104.

In operation 105, a fourth record in which a getting-in event has beentagged may be extracted.

When the process proceeds to operation 106, a record in which secondvehicle information included in the getting-in event tag of operation105 has been tagged may be extracted.

In FIG. 10 , operations 101 to 107 are shown, however, the disclosure isnot limited to the flowchart shown in FIG. 10 . For example, operation106 may be repeatedly performed until a getting-off event is tagged oruntil a record in which information about the second vehicle has notbeen tagged is found, as described above with reference to operation102. Accordingly, a continuous fifth record section may be extracted.

When tracking is completed, the route provider 113 may provide routeinformation including images and/or location information of framescorresponding to extracted records, in operation 107. The route provider113 may acquire, as the tracked result, two or more successive recordsfrom among the first record section extracted in operation 102, thesecond record extracted in operation 103, the third record sectionextracted in operation 104, the fourth record extracted in operation105, and the fifth record section extracted in operation 106, andcontinuously list images of frames corresponding to the records acquiredas the tracked result to provide the images to the terminal T, orcontinuously list location information of the frames corresponding tothe records acquired as the tracked result to provide the locationinformation to the terminal T. The location information of the framesmay be acquired from channel information of a camera that hasphotographed the frames.

The method according to an embodiment of the disclosure shown in FIGS. 9and 10 may be written in the form of a program that is executable on acomputer, and may be implemented in a general-purpose digital computeroperating the program by using a computer-readable recording medium.Examples of the computer-readable recording medium may include storagemedia such as magnetic storage media (for example, ROMs, floppy disks,hard disks, etc.), and optical recording media (for example, CD-ROMs,DVDs, etc.).

According to an embodiment, some of operations shown in FIG. 10 may beperformed. For example, in operation 101, when a request for a route ofthe first vehicle is received so that the process proceeds to operation102, records in which information about the first vehicle has beentagged may be extracted in chronological order, and whether the processproceeds to operation 104 may be determined according to a condition ofoperation 103. Because no getting-off event has been tagged in therecords extracted in operation 102, the process may fail to proceed tooperation 104 and operation 102 may be repeatedly performed. At thistime, a record without a tag of the first vehicle may be found. In thiscase, the process may proceed to operation 107. That is, in the currentembodiment, only operations 101, 102, and 107 may be performed.According to another embodiment, only operations 101 to 105 and 107 maybe performed.

When a route of the first person is requested from the terminal T inoperation 101, a flowchart resulting from replacing the person with avehicle in operations shown in FIG. 10 may be applied.

When records stored in the database 120 are collated regularly orirregularly to be backed up in a separate central database (not shown)according to an embodiment of the disclosure, the route provider 113 maygenerate and provide a route with reference to the central database inthe method shown in FIG. 10 .

The disclosure may be applied to a traffic information providing system.For example, the disclosure may be applied to a system for tracking asuspect or a suspected vehicle. The disclosure may be applied to imageprocessing technology for tagging images to create database.

While the disclosure has been shown and described with reference to theembodiments thereof, it will be understood by those of ordinary skill inthe art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. An object tracking method, comprising: generating an object tag for frames included in images captured by at least one camera photographing a traffic information, by recognizing at least one of a person and a vehicle from the frames; and generating an event tag referring to the object tag, the event tag comprising a getting-in event tag and a getting-off event tag, wherein the getting-in event tag is generated, when a vehicle and a person within a predetermined distance from the vehicle are recognized from a first frame, and the vehicle is recognized while the person is not recognized from a second frame following the first frame, and wherein the getting-off event tag is generated, when a vehicle is recognized while no person is recognized from a third frame and the vehicle and a person within the predetermined distance from the vehicle are recognized from a fourth frame following the third frame.
 2. The object tracking method of claim 1, wherein the generating the object tag comprises sampling the image into a plurality of sampled frames at an interval N of N frames, determining whether at least one of a person and a vehicle is present for each of the plurality of sampled frames, and generating a plurality of object tags for the plurality of sampled frames, respectively, according to the determined result, and wherein generating the event tag comprises generating a tag of at least one of the getting-in event and the getting-off event with reference to the plurality of object tags generated for the plurality of sampled frames, wherein the first frame and the second frame are successive in the plurality of sampled frames, and the third frame and the fourth frame are successive in the plurality of sampled frames.
 3. The object tracking method of claim 1, wherein the generating the object tag comprises generating a plurality of object tags for each of a plurality of frames included in the images.
 4. The object tracking method of claim 1, further comprising: generating a record for each of the frames included in the images, the record comprising the object tag; generating the event tag with reference to the object tag of the record; and storing the event tag in a record of a frame corresponding to the event tag.
 5. The object tracking method of claim 1, further comprising: receiving a request for a travel route of a first vehicle from a terminal; searching for records in chronological order in a database storing a plurality of records; and providing route information including at least one of an image and photographing location information corresponding to a continuous first record section including an object tag corresponding to information about the first vehicle, a second record including the object tag corresponding to the information about the first vehicle and a getting-off event tag, and a continuous third record section following the second record and including an object tag corresponding to information about a first person included in the getting-off event tag of the second record.
 6. The object tracking method of claim 1, further comprising: receiving a request for a travel route of a first person from a terminal; searching for records in chronological order in a database storing a plurality of records; and providing route information including at least one of an image and photographing location information corresponding to a continuous first record section including an object tag corresponding to information about the first person, a second record including the object tag corresponding to the information about the first person and a getting-in event tag, and a continuous third record section following the second record and including an object tag corresponding to information about a second vehicle included in the getting-in event tag of the second record.
 7. An object tracking apparatus, comprising: an object recognizer configured to recognize at least one of a person and a vehicle from frames included in images captured by at least one camera photographing a traffic information and generate an object tag for the frames; and an event recognizer configured to generate an event tag referring to the object tag, the event tag comprising a getting-in event tag and a getting-off event tag, wherein the getting-in event tag is generated, when a vehicle and a person within a predetermined distance from the vehicle are recognized from a first frame, and the vehicle is recognized while the person is not recognized from a second frame following the first frame, and wherein the getting-off event tag is generated, when a vehicle is recognized while no person is recognized from a third frame and the vehicle and a person within the predetermined distance from the vehicle are recognized from a fourth frame following the third frame.
 8. The object tracking method of claim 7, wherein the object recognizer is configured to sample the image into a plurality of sampled frames at an interval N of N frames, determine whether at least one of a person and a vehicle is present for each of the plurality of sampled frames, and generate a plurality of object tags for the plurality of sampled frames, respectively, according to the determined result, and wherein the event recognizer configured to generate a tag of at least one of the getting-in event and the getting-off event with reference to the plurality of object tags generated for the plurality of sampled frames, wherein the first frame and the second frame are successive in the plurality of sampled frames, and the third frame and the fourth frame are successive in the plurality of sampled frames.
 9. The object tracking method of claim 7, wherein the object recognizer is configured to generate a plurality of object tags for each of a plurality of frames included in the images.
 10. The object tracking method of claim 7, wherein the object recognizer is further configured to generate a record for each of the frames included in the images, the record comprising the object tag; wherein the event recognizer configured to generate the event tag with reference to the object tag of the record, and store the event tag in a record of a frame corresponding to the event tag.
 11. The object tracking method of claim 7, further comprising: a communicator configured to receive a request for a travel route of a first vehicle from a terminal; a route provider configured to search for records in chronological order in a database storing a plurality of records, and provide route information including at least one of an image and photographing location information corresponding to a continuous first record section including an object tag corresponding to information about the first vehicle, a second record including the object tag corresponding to the information about the first vehicle and a getting-off event tag, and a continuous third record section following the second record and including an object tag corresponding to information about a first person included in the getting-off event tag of the second record.
 12. The object tracking method of claim 7, further comprising: a communicator configured to receive a request for a travel route of a first person from a terminal; a route provider configured to search for records in chronological order in a database storing a plurality of records; and provide route information including at least one of an image and photographing location information corresponding to a continuous first record section including an object tag corresponding to information about the first person, a second record including the object tag corresponding to the information about the first person and a getting-in event tag, and a continuous third record section following the second record and including an object tag corresponding to information about a second vehicle included in the getting-in event tag of the second record. 